Skip to content

feat(orchestrator): in-memory queue + disk persistence (QueueStore)#607

Merged
gewenyu99 merged 1 commit into
experiment/orchestrator-01-bootstrap-gatingfrom
experiment/orchestrator-03-queue-persistence
Jun 18, 2026
Merged

feat(orchestrator): in-memory queue + disk persistence (QueueStore)#607
gewenyu99 merged 1 commit into
experiment/orchestrator-01-bootstrap-gatingfrom
experiment/orchestrator-03-queue-persistence

Conversation

@gewenyu99

@gewenyu99 gewenyu99 commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

In-memory task queue for the orchestrator. Synchronous and single-owner, no locking. Tasks carry type, dependencies, attempts, and a structured handoff for the next agent. Every transition rewrites .posthog-wizard/queue.json, the run's log today and the resume point later. nextRunnable returns every pending task whose dependencies are satisfied. Parallelism is decided by the task graph, not by a queue knob.

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci basic-integration
  • /wizard-ci error-tracking-upload-source-maps
  • /wizard-ci misc
  • /wizard-ci revenue

Test an individual app:

  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
Show more apps
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci error-tracking-upload-source-maps/android
  • /wizard-ci error-tracking-upload-source-maps/flutter
  • /wizard-ci error-tracking-upload-source-maps/ios
  • /wizard-ci error-tracking-upload-source-maps/next
  • /wizard-ci error-tracking-upload-source-maps/next-no-posthog
  • /wizard-ci error-tracking-upload-source-maps/node-raw
  • /wizard-ci error-tracking-upload-source-maps/node-rollup
  • /wizard-ci error-tracking-upload-source-maps/node-rollup-typescript-plugin
  • /wizard-ci error-tracking-upload-source-maps/node-webpack
  • /wizard-ci error-tracking-upload-source-maps/nuxt-3-6
  • /wizard-ci error-tracking-upload-source-maps/nuxt-4-3
  • /wizard-ci error-tracking-upload-source-maps/react-native
  • /wizard-ci error-tracking-upload-source-maps/react-vite
  • /wizard-ci error-tracking-upload-source-maps/rust
  • /wizard-ci misc/quack-quack
  • /wizard-ci revenue/stripe

Results will be posted here when complete.

gewenyu99 commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

Comment thread src/lib/programs/orchestrator/queue.ts
Comment thread src/lib/programs/orchestrator/queue.ts Outdated
Comment thread src/lib/programs/orchestrator/queue.ts
Comment thread src/lib/programs/orchestrator/queue.ts
Comment thread src/lib/programs/orchestrator/queue.ts Outdated
Comment thread src/lib/programs/orchestrator/queue.ts
@gewenyu99 gewenyu99 force-pushed the experiment/orchestrator-01-bootstrap-gating branch from f6620f8 to 51706a2 Compare June 16, 2026 21:48
@gewenyu99 gewenyu99 force-pushed the experiment/orchestrator-03-queue-persistence branch from 0b7c4fd to 1b9165b Compare June 16, 2026 21:48
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gewenyu99 gewenyu99 force-pushed the experiment/orchestrator-03-queue-persistence branch from 1b9165b to 0ccf004 Compare June 17, 2026 15:48

@edwinyjlim edwinyjlim left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

solid

@gewenyu99 gewenyu99 merged commit 7bea83c into experiment/orchestrator-01-bootstrap-gating Jun 18, 2026
12 of 13 checks passed
@gewenyu99 gewenyu99 deleted the experiment/orchestrator-03-queue-persistence branch June 18, 2026 13:52
@gewenyu99 gewenyu99 restored the experiment/orchestrator-03-queue-persistence branch June 18, 2026 13:56
@gewenyu99 gewenyu99 deleted the experiment/orchestrator-03-queue-persistence branch June 18, 2026 14:25
gewenyu99 added a commit that referenced this pull request Jun 18, 2026
…607)

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gewenyu99 added a commit that referenced this pull request Jun 18, 2026
…rd-orchestrator flag) (#606)

Experimental task-queue orchestrator, collapsed from an 11-PR stack into this single branch. Behind the `wizard-orchestrator` feature flag, off by default — with the flag off it is a no-op and the existing linear flow runs unchanged.

## Wizard PRs folded in (squash-merged into this branch)
- #607 — queue persistence
- #608 — orchestrator MCP tools: enqueue_task / complete_task / read_handoffs, with guards
- #609 — executor
- #619 — agent loader
- #620 — email targeting
- #635 — CI flag overrides
- #637 — ephemeral skills
- #638 — telemetry (responsiveness A/B spine)
- #639 — CI task filter
- #677 — run cache + responsiveness analytics
- #678 — CI health-check advisory

## Related PRs, other repos
- PostHog/context-mill#181 — `agents` content type: agent prompts plus the `basic-integration` step-skills, built and served alongside skills
- PostHog/wizard-workbench#1879 — CI flag-override and task-exclude dispatch inputs (dev/CI stack)

## Flag gating — why this is a no-op with the flag off
- `runProgram` forks on `isOrchestratorEnabled(flags)` = `flags['wizard-orchestrator'] === 'true'` (default false). Off, it runs `runLinearProgram`, the existing path, untouched.
- The shared `runAgent` threads the same flag through, so its orchestrator-specific message handling is gated too.
- The CI flag-override (`WIZARD_CI_FLAG_OVERRIDES`) is guarded by `NODE_ENV === 'production'` and stripped from published builds — a smoke test asserts the env-var names are physically absent — so it cannot flip the flag for real users.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants